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. , Besearch and etaluation are crucial elements of the 

curriculuB developnent process. Educational research properly 
includes « basic- nonevaluative* and "applied-evaluatite research." 
Each species should be judged relative to its purpose and the degree 
to which systematic investigation occurs through the application of 
the scientific aethod. Evaluative or noneyaluative- re search aust be 
justified in teras of the coaprahensiveness of the knowledge base 
related to the project's developaeht goals. Soae overlap of research 
purposes between evaluative^ and nonevaluative research aay occur with 
the iaproveaent of evaluative research aethodology. Belevancy to 
practical concerns aay be iaproved by exploiting the autual 
interdependence between evaluative and nonevaluative research. 
Practicing researchers aust recognize the inherent differences of 
purpose between evaluative and nonevaluative research, and support 
each according to its particular strengths, if the full potential of 
the total educational research eiterprise is to be realized. The 1 
Can Project, an instructional pragraa in physical education for 
trainable, aentally retarded, eleiientary school children, is 
presented as an esaaple of the relationship between evaluative 
research and curriculua developaent. (Author/SE) 
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OVERVIEW OF mim CURRICULUri DEVELDPfEJT Sm 

Introduction 

The focus of the I CAN Project' was to develop a set of Instructional 
physical education materials for elementary age trainable mentally Imjaalred 
children. The general aim was to develop a repllcable form of Instruction 
upon which a broad motor skill base and functional competence In selected 
games, sports and. activities of the culture, could be built. These skills, 
along with companion cognitive and affective learnings, are perceived to be 
enabling to I CANVs overall goal of soclo-lelsure competence. 

The general approach to the development of I CAN was guided by the 
considerations that the Instructional materials were to be: I) adaptable to 
the wide range of educational settings within which trainable mentally Impaired 
children are found, 2) objective oriented, 3) diagnostic and prescriptive, 
and 4) designed to operational Ue the concept of educational accountability. 

A further consideration was that these materials should not represent a 
sequenced set of lessons but rather a resource from which local school districts/ 
teachers could build programs more closely aligned with their unique objectives 
and resources. Concomitant with the resource material Idea was the requirement 
that a procedural model be developed to guide the program»s Implementation. This 
model takes the form of a teacher's manual that systematically guides a series 
of decisions, based upon local conditions, resulting In a unique and functional 
implementation plan. 

The Development of I CAN 

There Is a wide gap between the knowledge base, which relates to man and 
activity, and the teaching of physical education In fhe public schools (Vogel, 
1969). This gap can be attributed largely to the failure of curriculum 
development efforts to use the concept of educational engineering^ In developing 



The I CAN Project Is the popular name of the PROGRAMMATIC RESEARCH PROJECT 
In Physical Education for the Handicapped Child, Janet A. Wessel, Ph.D., 
Michigan State University, Project Director. 

See; Anderson, 1961; Cookingham, 1970; Gephart, 1970; Hamreus, 1969; and Mackle, 
1967 for a discussion of the importance of this concept In curriculum 
development. 



. their rnstruction materials. The gap between the knowledge base and the teaching 
of physical education to the trainable mental ly Impaired Is even more acute. For 
this reason; the I CAN Project focus became one of systematically converting this 
knowledge base Into a program which could^be tnplemented In the schools. This 
developmental approach can only be justified In situations where a large knowledge 
base Is available, and only when evaluative research, as treated later In this 
paper. Is an Integral part of the development effort. 

The evidence presented by the authors of footnote number 2, overwhelmingly 
supports the need for Intermediary stages In the research to practice effort as 
It relates to the Improvement of Instruction. The conversion of knowledge Into 
practice. Including the Intermediary stages, can be summarized In the following 

-Six steps: . ' 

1. Goal Identification and Objec^tlve Specification 

2. Description of Instructional , Design Specifications 

3. Development of the Instructlcjnal Prototype 

4. Evaluation of the Prototype ! * * 

5. ftodi float Ion of the Prototyp^ 

6. Recycl Ing 

Each of these steps Is further explicated |n Part II of this paper. 

Rational e for Materials Development Approarh 

The knowledge base related to activity and man Is rich with Information In 
growth and development, weight control, motor skill learning and performance 
maintenance and Improvement. Since most of this Information Is not utilized 
In current programs for the trainable, the approach to materials development 
was to: I) use what Is known In the subject field, 2) apply the newer pro- 
cedures suggested for Instructional design, which are briefly described In 
Part II of this paper, and 3) generate a product consistent with the general 
specifications described In the Introduction. As will be discussed later, this 
decision placed the research effort at the "applied" level In the form of 
evaluative research. This decision Is more clearly supported In the treatment ' 
of research, evaluation and curriculum development In Part II of the paper. 
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Introduction • \. . 

Th© following paragraphs attempt to portray the Interrelationships that exist 
between "research" and "evaluation" with I r. the context of curriculum development. 
The various types and characteristics o »d.icatlonal research are used In a 
discussion of t a specif contributions aid limitations of research In Improving 
the educational program at the classroom Icel. A poslt^on statement Is offered 
as.a summary of the Important Interrelationships which must be recognized If 
optimum progress In educational research Is to occur. 



The Conc ept of Educational Research 

The term "research" has been uSed to describe many forms of activity ranging 
from reviewing the literature through the rigorous form of laboratory application 
of the scientific method. Modifiers which are commonly associated with research 
activity Include: experimental, basic, pure, historical, descriptive, applied, 
library, evaluative, action, philosophical, cooperative, survey, etc. Admittedly, 
there Is redundancy within the narrow terms llsfed above. However, each In Its 
own context Is used as a functional descriptor In the application of the 
scientific method to approach and solve Issues of concern to the educator. It 
Is this common denominator, "application of the scientific method", which should 
be used In Judging any species of the more generic term, "research", with respect 
to Its position In the scientific community. 

Evaluation, like research, haS taken on many different meanings In recent 
years. Uses of the term range from the subjective Judgments made with or without 
-supportive data, through very sophisticated summatlve efforts which are similar 
In their precisions to controlled laboratory studies. Although evaluation, as 
In the previous summatlve characterization, may be correctly termed a species of 
research, this is not, and should not, always be the case. Rather the generic 
term, "evaluation" concerns the act of making a judgment of worth In comparison 
with some defined standard. Such a judgment can be made completely In the absence 
of the systematic Investigation which Is generally associated with research 



activities. Evaluative research, as characterized In the following paragraphs. 
Is an appropriate term for application of the scientific method In evaluating 
some educational product and represents a form of the "applied" species of 
research. 

Types of Educational Research 

Research types are commonly dichotomized Into basic and applied, each 
species connoting "goodness" or "badness" depending upon one's orientation or 
training. Using the criterion of systematic Investigation through appi l cation 
of the scientific method, both should be considered appropriate forms of research. 
"Goodness" and "badness" become appropriate only as descriptors of the degree to 
which Implementation was: 1) consistent with the stated, purpose, and 2) a 
rigorous application of the scientific method within conlextua I constraints. 

Since this paper considers research, evaluatloh and curriculum development, 
the terms evaluative research and non-evaluatlye research w 1 1 1 be used. Evaluative 
research wlM be characterized as a form of applied research and non-evaluative 
research will be used as a descriptor for those species of research conmonly.. 
termed basic, pure or laboratory. Briefly defined, non-evaluative research " 
refers to those forms of Inquiry which have as their major purpose the expansion 
of theory through hypothesis testing and Involve carefully controlled designs 
to obtain general Izable results. Evaluative research holds as Its major purpose 
generating data for descriptive and judgmental uses which are not necessarily 
conclusive or general izable. 

Characteristics of Evaluative and 
Non-Evaluative Research 

The overriding characteristic of sound educational research should be con- 
sidered to be systematic Investigation through the application of the scientific 
method. There are, however, different characteristics within the various species 
of research which should be noted; In Figure I below educational research is 
portrayed according to Its two types: evaluative and non-evaluative. Con- 
trasting characteristics are presented according to several important categories. 
(Note: Characterizations such, as those syggested below are necessarily an 
over-simplification and should be Interpreted within this constraint. 



Category 



Goal 

Objectives 

General I zab 1 1 Ity 

Control 
Variables 

Utilization 
Tlmelessness 



Oes 1 gn 

Validity 

I nvest i gat I ve 
Technique 



educational research charact'^stics 

research' type 

— Non-Evaluative 
"Basic" 



Evaluative 
"Applied^' 



"Typical" Characteristics 



Knowledge generation 
(hypotheses-theory- 1 aws 

Increaised understanding 
regardless of value for 
social change 

High general t zab 1 1 Ity 



Control /er all ^ut 
tndepenc nt variables 

Abstract »vthe concept 
Is the variable df 
Interest 

Future and/or 
Immediate 

General Izable across 
time 

High Internal validity 
Low external valld/lty 



Testing the application^ 
of knowledge 

Information for program 
planning, development 
or operation 

Low general I zab 1 1 Ity - 
Data for description 
and judgment 

Limited control over 
Intervening variables 

Concrete - the observable 
phenomena Is the variable 
of Interest 

Immediate 



Time specific 



Low Internal validity 
High external validity 



Application of the scientific method 



Figure 1. Educational Research Characteristics 



CURRICULUM DEVElDPfOT 



Introduction ' . ' 

The tartr curriculum development has commonly been used to describe those 
activities which convert theory into practice. Although research^to-practlce appears 
to be a reasonable transition It is quite Impractical without Intermediary steps. 
As one might suspect, there Is no one-to-one correspondence between laboratory 



phenomena (with all their necessary controls) «nd classroom phenomena (with their 
characteristic lack of controls) (Cooklngham, 1970; Twelker, 1969; Mackle and 
Chrlstensen, 1967). Re ^nt thinking In curriculum development has 
advanced the concept c .e educational engineer or architect (Melton, 1959; 
Taba, 1962; Gi^ba, 1965; Anderson, 1961; GepHart, 1970; Cooklngham, 1970). As 
characterized by Figure 2, one could represent this common practice by portraying 
a wide canyon. On the ne side Is a large knowledge base in the form of sub- 
stantiated hypotheses, theories and laws, and across the canyon educational practi 
Is portrayed. Success at crossing such a gap without some sort of "bridge" must 
be considered quite remote (Ausubel, 1953; Cooklngham, 1970; Gephart, 1970; Mackle 
1967). With the aid of- the engineer, however. It becomes a matter of using. the 
knowledge base to generate a set of specifications for building a bridge, con- 
structing the bridge, and thdn evalua'ting Its utility. Such an undertaking Is 
descrlptlve-^f the modern curriculum development process (Twelker, 1969; Hamreus, 
1969; Davis, 1972; Cooklngham, 1970). 

CURRICULUM DEVELOPMENT 




EDUCATIONAL EKGINEERINC 




Practice 



Figure 2. Educational Engineering Portrayed 
as a Bridge Between the Knowledge Base 
and Instructional Practice 



The Curriculum Development Process 

As suggested above the process of curriculum development must extend beyond 
the two sVages; research -to- practice. A comprehensive model of curriculum 
development will address each of the following steps: 

1. Goal Identification and Objective Specification 

2. Description of Instructional Design Specifications 

3. Development of Instructional Prototypes 
'4. Evaluation of the Prototypes 

5. Modification of the Prototypes 

6. , Recycl i ng 



Goal Identlflc^tton and Ohl^tlve Speclflr.Mnn ,s a social process con- 
ditioned by political and economic constraints which defines tj,e purpose for 
«htch the Instructional progran. Is being developed (Hamreus. I969J. It Identifies 
what is to be taught and reflects both the needs and values of society; P™cedural 
considerations used to Insure relevancy of purpose Include library, survey, and 
descriptive research and task description and task analysis. 

^"^ P^scriptlon of Instructional n^Mnn Sne. I fl. ,s 3 ^ask of the 
educational engineer (Twelker. 1969). He must look at the goaKs, of the pr^^gram. 
extract the related hypotheses, theory and laws from the knowledge base and propose 
a blueprint which guides the development of the InstrucYlonal pFS^type It Is 
at this stage that non-evaluative. or basic, research is most closely related 
to the curriculum development process. 

Twelker further divides Instructional specification Into two major divisions- 
I specifying the instructional sequence and 2) specifying the instructional con- 
ditlons. Briggs (I96S) suggests- that sequencing Is a nore powerful influence In 
determining performance levels than how the competencies are taught. Further 
there is support for the hypothesis that mastery of lower order objectives ' 
positively Influences learning of -higher order objectives (Gagne. 1965 1962- 
Gagne, et^,. ,962; and Gagne and Paradise. ,961). Specifying Instructionli 
conditions combines a systematic use of the knowledge base with experience and 
artfullness", plus the considerations of learner characteristics. Instructional" 
context, stimulus situations, learner responses an. feedback situations (Twelker. 
1969). It would be most desirable If the suggested mix of knowledge, art and 
experience were prinarlly dependent-on the knowledge base; however, many gaps 
exist, causing experience and art to play major roles In the generation of 
instructional specifications. From the standpoint of curriculum development 
the existence of these gaps should be considered as strong rationale for the 
Inclusion of "basic" or non-evaluative research In the total educational research 
effort. 

, ,, P^''^'°P'"«"t of th» instruction.! Protnty p.. involves the act of systematically 
following the Instructional specifications to produce the substance, form and order 
of the Instructional materials. It Involves the process of translating a set of 
written statements (specifications) into an Instructional product. However, clarity 
and completeness of Instructional specifications do not guarantee a quality 
product. Of equal '^ortance are the dl ff leu It decisions concerning conditions 
stimuli, fon.at. strategies, etc.. which require the abilities of talented developers. 
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When ajmplete, these decfsfons will, to a degree, reflect the accurate organization 
of the design specifications and the knowledge base from which they originated. 
Since the curriculum prototype Is necessarily removed from Its underlying knowledge 
.base, a to out, built Into the developroeot process, becomes a necessary step 
for quality control. No matter how sophisticated the developmental process, 
developers have learned to expect about 60$ effectiveness during the first 
Implementation effort (Sorenson, 1970). 

The primary concern during evaluation of the prototY pA is to determine, fn 
a systematic fashion, the weakness or deficiencies of the curriculum produtt. 
Since this process Is formative. Implying modification, evaluators must be con- 
cerned not only with what I s> deficient, but also why, (Hastings, 1966). The 
process of formative evaluation Is the suggested methodology (Scrlven, . 1967; 
Cronbach, 1963; Sorenson, 1971; Alkln, 1970; Stake, 1967). Concerns of rellvancy, 
feasibility and usability, as well as effect, are appropriate aread..of -invest I- 
gatlon. Small sample, cyclical studies which examine various elements of the 
program and are conducted within the context of Intended use should proceed 
broader field tests (Hamreus, J 969). 

The appropr later standards for Judging a program's effectiveness have been 
•a point of disagreement In the field of evaluation. Cronbach (1963) favors 
comprehensive description and Scrlven (1967) demands comparison. In the case 
of the formative evaluation of prototype materials, Hamreus (1969) clarifies 



the issue: 



When a new system has been sufficiently developed and defined 
and in use over a period of time such that Its accomplishments can 
be predicted, then comparative analysis with another established system 
. are appropriate. When the system is still new and essentially unified 
the on y basis for Judgment lies In how well It attains the objectives 
for which It was created [p. 22]." jwwmvcs 

Evaluation of an Instructlon^prototype moves from Infonnal, Judgmental procedures 
toward more formal, evaluative Research methodologies during the field test. Its 
relation, to theory Is primarily one of hypothesis generating and can only provide 
support for theory building in unusual situations where tightly controlled designs 
are possible. 

'^^^ ^Edification of Prototvpa f/^f^rtpte. should be executed on the strongest 
data base possible. Generally speaking, all recorded deficiencies sbould be 
supplemented with alternative suggestions accompanied by rationale that explain 



Why they may be rmprovements. ReWslon criteria should be established and app'led 
to each suggested change. Judgments must be made regarding whether or not th& 
suggested modification Is consistent with the original Instructional specifications. 
In Instances where Inconsistency Is apparent, both the suggested change and the 
Instructional specification should be reviewed for Inaccuracies. 

BecjtcJini represents the final stage of curriculum development, and Is 
Initiated for all or part of each Instructional product based on whether or not ' 
the stated object I vas are attained. 

It Should be noted that each of the curriculum development steps Is not 
a discrete process but rather one of overlapping complexities. It Is Important 
however, that such steps be clearly In mind If curriculum development Is to ' 
occur as a systematic conversion of the knowledge base Into Instructional 
practice. 



- IBE»ip« EVlMTIOfllH CURRICULUM 

KVEUFBff - A POSITim STAIBBJT 

Need for Both Evaluative and 
Non-Evaluative Research 

The preceding paragraphs suggest the need for both non-evaiuative and evaluative 
educational research in the currlculun development process. Non-evaiuative research 
Characterized by Its focus onhypothesis testing, theory expansion, geneniilz- 
abllity, high internal validity and tin«lessness results In products which can 
be utilized by educational engineers for developing the specifications from which 
educational programs can be developed. Evaluative research. characterUed by 
high external validity, low general izabi I Ity. information for decision making 
immediate utility, time specificity and hypothesis generation is defined as an 
integral part of the cuVricuium development process. 

Even though both evaluative and non-evaluative forms of research are important 
to curriculum development, the developer must be able to Justify the degree to 
which he engages In each, in those instances where a large knowledge base exists, 
there appears to be less Justification for the conduct of non-evaluative research 
within project activities. Rather, emphasis should be on the curriculum develop- 
ment process, which includes generous amounts of evaluative research activity as 
a means of quickly incorporating that which Is known Into the realm of educational 
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practice. Who, tho knowledge base Is shallow, forcing «a„y tenuous assu«ptlons 
at the Instructional specification level and where sufficient time and rosourc;«s 
are available, non-evaluative or laboratory type research efforts n»y be easily ' 
Justified. Curriculun develop«»nt Is a long, difficult and expensive task which 
be conducted quite independent of sound assumptions or prudent instructional 
^specifications. However, effort expended in developing a curriculum kII I not 
correct basic errors In des^n fashioned frora faculty assumptions. 

y The Nefid for f^on peratfon Bfttw^an 

Evaluative and Non-Evai uattve Researrhore ' ; 

As one studies the characteristics of evaluative and non-evaluative research 
portrayed In Figure I, It becomes quite aparent that researchers In both areas 
™«st put aside their allegations of goodness and badness, based on the type of 
research conducted. Characteristic comments of. "no control" or "too general" 
leveled at applied research and. "too sterlW" or "unrealistic" leveled at basic * 
research are entirely Inappropriate within the broader context of educational 
research. The need for systematic investigation within the species of research 
must be used as the criterion denoting quel ity rather than the purpose and con- 
straints within which the research type Is conducted. 

Perhaps further Justification for acknowledging the Importance of each type 
of research Is their mutual dependence for elevating both the relevancy of 
baric research and the quality of educational practice. In this age of "tight 
monies" and practical priorities the mere fact that a curriculum development act 
has been funded Is an Indication of need or relevancy. Evaluative research. In- 
herent In good curriculum development, becomes a rich source of relevant hypothesi. 
Which Should be Investigated within the context of tightly Control led. non-evaluative 
research. 

As evaluative research methodology Improves, It sea^s entirely possible that 
•ts goals may be expanded to Include the provision of data to support theory- 
related hypotheses. Through what Rosenshlns (1972) refers to as ««,nltorlng the 
degree to which the Instructional product is I^lemented as intended, information 
concerning the wh^ of Identified effects may be analyzed through evaluative 
research. Variables of Interest could extend beyond the traditional antecedent 
and transaction variables described by Stake (1967) (although these are highly rele- 
vant) to ones related to the instructional specifications used In step two of 
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the currtculum development process. Depending on the degree to which certain 
^theoretical premises are. operational I zed In the Instructional product and applied 
. In the Instructional setting (descrlb^^ by the nonltoring pn>cess), one could 

test their effects In a more external ly valid manner. lnfoni,ation of this type 
_could be particularly helpful If such experiments were replicated wiVh the 

systematic varying of Interveing variables. 

Summary 

Research and evaluation are crucial eleoients of the curriculum development 
process. Educational research properly Includes "basic - non-evaluative" and 
"appl led - evaluative research". Each species of research should be Judged 
relative to its purpose and the degree to which systeratic investigation occurs 
through the appl Icatjon of the scientific i^thod. Evaluative or non-evaluative 
research, as a legitlmat?.,cjirrlculun development project activity, must be . . 
justified in terms of the comprehensiveness of the knowledge base related to the 
project's, development goals. Some overlap of research purposes between evaluative 
and non-eva I uatlve research nsy occur with the improvement of evaluative r^rch ■ 
methodology. Relevancy to practical concerns may be imprx>ved by exploiting the 
««t«al Interdependence between evaluative and non-eva I uatlve research. Practicing 
researchers must recognize the Inherent differences of purpose between evaluative 
and non-evaluative research, and support each according to Its particular strength- 
.f the- full potential of the total educational research enterprise Is 1o be ' ' 
realized. ; - 
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